N95- 15974 


APPLIED VIRTUAL REALITY AT THE RESEARCH TRIANGLE INSTITUTE 

R. Jorge Montoya 
Senior Research Engineer 
Manager, Virtual Reality Group 
Research Triangle Institute 
P.O. Box 12194 
RTP, NC 27709-2194 


ABSTRACT 

Virtual Reality (VR) is a way for humans to use 
computers in visualizing, manipulating and interacting 
with large geometric databases. This paper describes 
RTTs VR infrastructure and its application to marketing, 
modeling, architectural walkthrough, and training 
problems. VR scientific integration techniques used in 
these applications are based on a uniform approach 
which promotes portability and reusability of developed 
modules. For each problem, a 3D object database is 
created using data captured either by hand or 
electronically. The objects' realism is enhanced through 
either procedural or photo textures. The virtual 
environment is created and populated with the database 
using software tools which also support interactions 
with and immersivity in the environment. These 
capabilities are augmented by other sensory channels 
such as voice recognition, 3D sound, and tracking. Four 
applications are presented: a Virtual Furniture 
Showroom, Virtual Reality Models of the North 
Carolina Global TransPark, a Walkthrough of the 
Dresden Frauenkirche, and the Maintenance Training 
Simulator for the National Guard. Degree of realism 
and update rate requirements for these applications 
posed significant implementation challenges which were 
met in every case. These applications demonstrate the 
viability of VR and show great promise for VR as a cost 
effective marketing, training, and teaching tool. 

INTRODUCTION 

Virtual Reality is an exciting new approach to human- 
computer interactions. Based on long-established 
computer graphics techniques and benefiting from 
recent advances in computer hardware and software, this 
technology supports the creation of and interaction with 
"worlds" which are either faithful replicas of existing 
ones or evoke the existence of yet-to-be-created ones. 
In its purest form, VR is the presentation of and 
interaction with a synthetic, computer-generated, 3D 
world, so realistic that the user feels as if he/she were 
experiencing reality. Over two and one-half years ago. 


RTI made a market and technology analysis that 
concluded that virtual reality was a technology poised 
for transfer from basic research laboratories to applied 
research institutions such as RTI. Based on this 
conclusion, RTI developed a business plan that defined 
the VR market segment which it would pursue, 
identified areas in which it would invest internal 
research and development (IR&D) funds, and mapped 
out the hardware and software configuration necessary 
for the development of an advanced VR laboratory 
which would serve as the foundation for its work in 
Virtual Reality. 

The market analysis recommended that RTI concentrate 
in a market consisting of architectural walkthroughs, 
marketing, rapid prototyping, and tr ainin g applications. 
These application areas would allow RTI to take 
advantage of its strong multidisciplinary backgro un d 
and provide value added as scientific and engineering 
integrators with the appropriate mixture of technology 
and domain experts to accomplish a specific job. 

This paper describes the RTI VR Laboratory 
infrastructure and several VR projects which the 
institute has undertaken. The projects described in this 
paper are examples of the application of Virtual Reality 
to marketing (a Virtual Furniture Showroom), p lanning 
(Virtual Models of the North Carolina Global 
TransPark), architectural walkthrough (a Walkthrough 
of the Dresden Frauenkirche), and training (A 
Maintenance Training Simulator for the National 
Guard). 

The paper includes a description of the methodology 
used to implement the various applications and a 
discussion of the system performance achieved in each 
of these applications. 

THE RTI VIRTUAL REALITY LABORATORY 

RTI is an independent, not-for-profit corporation 
founded in 1958 by the University of North Carolina at 
Chapel HSU, North Carolina State University, and Duke 
University. RTT conducts applied and basic multi- 
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disciplinary research for governmental agencies and 
commercial clients. 

RTI has developed a nationally recognized program in 
computer graphics applications over the past twenty 
years [1, 2, and 3]. The emphasis of RTTs work in 
computer graphics has evolved and advanced as 
hardware capabilities have improved and software tools 
have become more sophisticated [4 and 5]. In keeping 
with its tradition of conducting advanced 
multidisciplinary applied research, RH has established 
a state-of-the art Virtual Reality laboratory with an 
investment of well over $1,000,000 over the last two 
years. 

Hardware: 


The VR laboratory infrastructure is shown in Figure 1. 
The backbone of the laboratory is a network of 
computers that includes the full range of VR 
capabilities. The computing environment consists of 
platforms ranging from PCs (Pentiums, 486s, and 
Apple’s Quadra 840 AVs) to the Silicon Graphics 
deskside Crimson Reality Engine and the rack mounted 
Onyx Reality Engine2. It also includes two IBM RS- 
6000s, a model 320 and a model 570. The environment 
also supports full immersion with a Virtual Research's 
EyeGen3 head mounted display (HMD) and see- 
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Figure 1 RTI VR Laboratory Infrastructure 


through immersion with Stereographies’ CrystalEyes 
shuttered glasses; tracking with a Polhemus magnetic 
tracker and an acoustic Logitech tracker; navigation with 
a joystick or mouse; a stereo projection capability using 
the VREX-1000 system; a wide range of modeling and 
rendering software environments; and speech 
recognition and sound output capability. 


The Crimsom and the Onyx are used for high-quality, 
high-performance graphics rendering. The Pentium PCs 


equipped with optional graphics cards are used for low- 
end graphics rendering. In addition, as shown in the 
figure, this core VR computational and graphics facility 
is networked to the rest of the computer infrastructure at 
RTI and to extramural computer facilities such as the 
supercomputer from the Microelectronics Center of 
North Carolina. 

Software: 

The programming environment in the laboratory is Unix 
based. The SGI graphics workstations operate under the 
Irix operating system. Software packages used at RTI 
include Performer, Inventor, and Explorer. In 
particular. Performer provides a high-level application 
programmers’ interface (API) for rendering the high- 
quality images which are characteristic of high-level VR 
applications. The IBM RS-6000 computers operate 
under AIX. The Apple Quadra PCs operate under the 
Apple OS operating system and the IBM PCs operate 
under the DOS operating system. 

The programming model for the development of VR 
applications is illustrated in Figure 2. Using a variety of 
modelers and format translators, RTI has developed the 
capability of providing cost-effective VR solutions and 
to deploy them on the most appropriate platform from 
PCs to Silicon Graphics. One such approach is based on 
a rapid VR prototyping capability developed by RU 
under an IR&D project and based on the low-cost 
modeler Virtus Walkthrough Pro®. Annotated 
polygonal databases are created in the modeler, either 
from electronic data or from drawings, and the results 
are transferred electronically into one of the Silicon 
Graphics workstations where the virtual environment is 
composed by the application of shading, textures, and 
light sources. The textures are derived from 
photographs of real objects scanned into the system 
through one of the IBM PCs. Textures can also be 
implemented procedurally. Once the environment is 
completed, interaction with the model is added using 
VR-DECK [6], a C-based software package developed 
by IBM T. J.Watson Research Center, in one of the RS- 
6000s. This package supports the instantiation and 
interaction of modules dedicated to specific functions in 
support of interactivity with and immersivity in the 
virtual environment. Thus, for example, there are 
modules dedicated to the capture of tracking and 
navigation data. These modules produce events which 
are used by a graphics-generating module to control the 
position and view of the camera model "searching” the 
data base. Additional features, such as speech 
recognition, sound generation, object behavior, etc., are 
available or can be added through additional modules. 
These modules can be distributed within one or many 
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different workstations in the network, including the 
Reality Engines. This provides the ability to match the 
system resources to the application requirements. 



Figure 2 Conceptual Approach to the 
Implementation of the Virtual Environment 

Another application development path in the laboratory 
uses Autodesk's AutoCAD to capture or generate data 
for the models, 3D Studio to generate the models, and 
Sense8’s WorldToolKit for development of the 
interactions with and immersivity in the resulting virtual 
environment. This environment is targeted toward the 
development of environments to be deployed on PC 
platforms. Current efforts have as a goal the creation of 
a seamless development environment in which models 
created by either modeler or from other non-RTT 
modelers (such as Intergraph's or ProEng) can be used 
in either the low- or high-end environment easily and 
cost-effectively. 

In addition, the modular software system supports head 
tracking and stereo viewing to provide a trackable, 
immersive capability in the application. Further 
interactions with the virtual environments is achieved 
through the . use of a speech recognition system 
operating in the RS-6000. Plans call for the development 
of software to support the integration of a 3D sound 
system and natural language processing into the 
applications as necessary. 


VR PROJECTS 

The Virtual Furniture Showroom: 

For the Furniture Manufacturing and Management 
Center at North Carolina State University, RTI created 
and demonstrated the Virtual Furniture Showroom. 
This was a technology demonstration project for the 
furniture industry. Using drawings and sketches 
provided by American Drew Lie. and photographs of 
the real pieces of furniture, RTT developed virtual 
models of American Drew's Hancock Cherry bedroom 
collection, replicated synthetically the room in which the 
real collection was, and arranged the virtual collection 
and accessories in the room. 

Interactivity with the furniture took on several forms. 
The visitor to the virtual showroom, donning a tracked 
HMD, could navigate over to a specific piece of 
furniture by gazing towards the piece and using a four 
-function joystick to walk over to or away from the 
piece. Then, by clicking on one of the other two buttons 
of the joystick, he/she could get a description of the 
piece through the earphones of the HMD. The 
application also supported the picking and moving of 
pieces of furniture as well as the changing of the finish 
on selected pieces within the collection. 

Figure 3 illustrates the resulting virtual environment. 
This application was developed using Virtus 
Walkthrough Pro™ . Interaction and immersivity was 
obtained through VR-DECK. Rendering was done in an 
SGI Crimson and presented in a Virtual Research 
Eyegen3 HMD. Average scene complexity was on the 
order of 4,000 polygons with an average pixel depth of 
5. The demands for high quality models required a 
large number of textures which, in some instances, 
overloaded the texture memory available in the 
Crimson, thus affecting rendering performance. 
Stereoscopic update rates on the order of ten per second 
were achieved when texture paging was not a factor. 
Update rates dropped substantially from this number 
when texture paging was a factor .The virtual showroom 
was demonstrated at the High Point, North Carolina, 
Fall International Home Furniture Market in October of 
1993. The virtual showroom exhibit was located near 
the room with the real collection. This provided a way 
for the visitors to do a comparison while their 
impressions of the virtual room were fresh in their 
minds. Based on the reaction of the majority of the 
visitors, the virtual exhibit was a success. From a 
technology standpoint, it was concluded that VR 
technology was viable for the furniture industry in at 
least two areas: rapid prototyping of pre-market 

collections and interactive electronics catalogue of 
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collections at high-end furniture stores. Use of this 
technology in furniture design and in many other 
applications should be feasible within the next five years 
as the cost of the technology continues to decrease and 
its performance continues to increase. 


Figure 3 The Virtual Furniture Showroom 

A Walkthrough of the Dresden Frauenkirche: 

For two-hundred years, the Frauenkirche (Church of 
Our Lady) stood majestically over Dresden, Germany, 
as a magnificent example of Baroque architecture and an 
important expression of the Lutheran faith. On 
February 15, 1945, the church collapsed as a result of 
the intense heat produced by fire storms which resulted 
from extensive Allied bombings of the city. Today, 
efforts led by the Foundation for the Reconstruction of 
the Dresden Frauenkirche are under way to rebuild the 
church. 

For IBM Germany, RTI developed an interactive and 
immersive walkthrough of the Dresden Frauenkirche. 
The goal of the project is to use the VR walkthrough as 
an exhibit where people can "visit” the church while it 
is under reconstruction. Figure 4 illustrates the interior 
of the Frauenkirche as it was in February, 1945, or as it 
will be in February, 2006. Thus, with the aid of Virtual 
Reality the visitor can step forward fifty years into the 
past and visit the magnificent Frauenkirche. 

The interactive model was based on an animation model 
developed using the TDI modeling package [7]. This 
model had been derived from an original architectural 
model of the church done in the CAD software package 
CATIA™. The system used to implement the interactive 
and immersive walkthrough of the Frauenkirche ’ 


consists of two workstations, an IBM RS-6000/650 and 
a Silicon Graphics Onyx Reality Engine, networked to 
accomplish the task. The tasks included tracking the 
orientation and navigation of the user, "the visitor" to 
the church, generating events resulting from the 
interpretation of the position data, updating the camera 
view of the database, generating a stereo pair of such 
view, and driving the two eye views of the head 
mounted display (HMD). 

RTT scientists converted the animation model of the 
Frauenkirche (in TDI format) into a format compatible 
with the Performer™ . Once this was done, application 
modules were implemented in VR-DECK and 
instantiated in the RS-6000. These modules included 
support of the head tracking operation, navigation, by 
means of the special joystick, and graphics. The latter 
invoked the rendering software developed for this 
application in the SGI Onyx. These modules were 
interconnected and activated in the RS-6000 according 
to the VR-DECK application protocol [8]. 

In particular, the graphics rendering module has the task 
of creating a view of the pictorial database (the church) 
as dictated by the head orientation and the user's 
position sensed by the tracker and indicated by the 
joystick, respectively. In the case of an immersive 
environment, this module also has the task of generating 
a stereo pair to support the presentation of a stereoscopic 
display in the HMD. 

The interactive model of the Frauenkirche is one of the 
most complex models used to date to generate a virtual 
reality walkthrough application. (See Figure 4). The 
model consisted of 165,000 polygons, it made use of 12 
textures in over twenty locations of the church, and was 
lit by five light sources. Scene complexity varies from 
approximately 80,000 polygons in the altar area to 
20,000 polygons toward the back of the church with an 
average pixel depth of 4. Update rates varied depending 
on the direction of view with views of the altar updating 
at 3-5 frames per second (in stereo) and views of the 
balconies and the back of the church updating at 10-20 
frames per second (in stereo). 

Virtual Reality Models of the North Carolina Global 
TransPark: 

For the North Carolina Air Cargo Airport Authority, 
RTI developed a series of immersive and interactive 
virtual models of the North Carolina Global TransPark. 
The NC Global TransPark is a bold initiative of the state 
of North Carolina to develop a multi-modal 
transportation facility built around the existing jetport in 
Kinston, NC. A development plan for the TransPark 
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has just been completed. This plan calls for the 
development of the facility in stages and the models 
'which RTI has implemented represent the various stages 
of development. Thus, the immersive models show the 
evolution of and allow planers to visit the concept from 
the present day to the project's conclusion in the next 
century. Figure 5 illustrates the resulting virtual 
models. 

The three virtual models of the TransPark were built 
using the Virtus-based rapid prototyping capability. 
Unlike the case of the furniture showroom, however, 
existing CAD data was used extensively in the creation 
of the models. The first model shows the Kinston, NC, 
and its significant surroundings. Model 2 shows the 
intermediate development of the TransPark which 
includes a 13,000 feet cargo runway, a central cargo 
facility, a cargo transportation system, and assigned 
areas for the location of various activities such as office, 
research, and light industrial activities. Model 3 
includes the complete vision of the TransPark which 
contains a second runway parallel to the original one. A 
control panel associated with each model allows the 
visitor to look at the existing wetlands and to visualize 
how they will change as die development takes place. In 
addition, time of day and visibility conditions can also 
be controlled through the panel. Included in all models 
are ’’dumb" agents representing activities which will 
take place in the Park. These include airplanes taking 
off and landing, trucks moving along highways, and a 
train moving along the railroad tracks. 

There are several display options associated with this 
application. The models can be shown in a stereoscopic, 
augmented reality mode either on a monitor screen 
using shuttered glasses or on a projection screen using 
passively polarized glasses with no head tracking and 
mouse-based navigation; or they can be shown in a 
HMD with head tracking and joystick-based, direction- 
of-gaze navigation. 

The models range in scene complexity from 4,000 to 
10,000 polygons with an average pixel depth of 3. 
Stereoscopic update rate performance varies from 10 to 
20 updates per second. Texture paging was not a factor 
in this application and we also were able to use level-of- 
detail models to optimize performance. 

Maintenance Training Simulator-National Guard 
(MTS-NG): 

For the Advanced Research Projects Agency, RTI has 
designed and implemented an advanced training system 
for home-station tr ainin g of National Guard tank 
mechanics. The maintenance training simulator for the 


National Guard (MTS-NG) is a computer-based 
instructional system which uses Virtual Reality as the 
human-computer interface between the trainee 
(mechanic) and the training system, significantly 
extending training to personnel at sites without 
equipment. The MTS-NG integrates VR, multimedia 
and instructional technologies to provide tr ainin g to tank 
turret mechanics (45T) to perform diagnostic and 
maintenance on M1A1 Abrams Tank and M2A2 
Bradley Fighting Vehicle. Figure 6 shows the MTS-NG 
development team testing the various stages of the 
advanced instructional system. 

This application has been implemented in a 90 MHz 
Pentium PC equipped with a SPEA Graphiti Series Fire 
graphics board and a StereoGraphics Corporation’s 
CrystalEyes PC for stereoscopic image generation and 
viewing. The software development environment 
consists of Autodesk’s 3D Studio modeler for the 
generation of the databases, Sense8's WorldToolKit for 
the building of and interaction with the virtual 
environments, and MicroMedia's Authorware for the 
generation of the courseware. 

The courseware includes the lessons used in the 
Regional Training Site (RTS). The courseware launches 
the virtual reality applications when appropriate to the 
purpose of the lesson. These virtual environments 
include navigation through and interactions with solid 
models of the two vehicles as well as cutaways of their 
interior showing all the Line Replaceable Units (LRUs). 
They also include the ability to interact with the interior 
of the gunner’s compartment and of the driver's 
compartment. Using this interactivity, the student may 
select any of the LRUs in either compartment for closer 
inspection and in turn switch switches, rotate knobs, 
etc., which can be used in performing diagnostic tests 
when used in conjunction with the Simplified Test 
Equipment (STE) and under the supervision of the 
training module. The training module guides and/or 
monitors the students’ progress in diagnosing a fault in 
the vehicle. The student uses military training manuals 
to perform a series of interactive tests in a mann er 
identical to the ones in the real vehicle. 

Scene complexity varies from about 2,000 polygons for 
the external 3D views of the vehicles to 15,000 
polygons for the interior view of the gunner's 
compartment with an average pixel depth of 3. These 
are textured polygons. Stereoscopic update rates vary 
from 1-2 frames per second for the most complex scenes 
to 10-15 frames per second for the less complex scenes. 
Texture memory available in the FIRE graphics board (8 
MBytes) accommodates the textures used in this 
application. Anticipated hardware and software 
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performance improvements will improve current 
performance. Also, if necessary, improved performance 
can be attained with a higher-performance platform. 
This will require a tradeoff between cost and 
performance since one of the design goals was to deploy 
this system for less than $10,000 per copy. Total system 
cost is $9,600 per unit. 

CONCLUSIONS 

Our work and that of many other applied researchers 
have demonstrated that VR is a viable technology with 
serious applications in areas other than low-end 
entertainment (arcades) and high-end simulators. When 
should one use VR? When the value of applying it 
exceeds the cost of developing it and also when it 
supports, enhances, and improves current or anticipated 
practices. It is our experience that development of a 
detailed requirements definition as a first step of a VR 
project leads to the development of cost effective 
solutions of VR problems. The bottom line is that 
virtual environments should not be more real t han 
necessary for the application. Hardware and software 
performance improvements will continue to support 
more and more sophisticated applications of VR at 
lower and lower cost. As it was with PCs in the early 
1980s, this will lead to the democratization of VR and 
the proliferation of its applications. We also anti cipate 
wide use of VR technologies in the rapid product 
prototyping arena. 
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Figure 4 VR Model of the Dresden Frauenkirche 
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Figure 6 Advanced Instructional System for the Maintenance T rainin g Simulator for the National Guard 
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